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Reported in' this document a£e findings of a / 
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application process >ior government-sponsored school meal behefits-and- 
an examination -of low-cost quality assurance procedures implemented 
in Phase I of the Income Verification Pilot Project ( IVPP)'. 
Discussion is based on an analysis of two of; three data sources used 
to evaluate Phase I quality assurance 'procedures: .(1) meal benefit 
progranPeJ-igibility and participation records, and (2)' abstracts of 
meal benefvit applications. Six chapters respectively provide a review 
of the background and current status of Phase I; an analysis of the 
effects of congressionally mandated application changes and . - 
experimental quality assurance procedures on program /eligibility and 
dosts to the Federal government; an analysis of the .effects of 
congressionally mandated application changes and experimental quality 
assurance procedures on meal benefi^ applications;/^ summary of , . H - 
findings; a discussion of implications of findings-; and appendices 
describing the statistical methods used in estimating effects and^ 
providing sample meal benefit applications'. (RH) 
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. EXECUTIVE SUMMARY _..;< ; • i . ; ,* . t 
■ • 'V . 1 '. ! ' ' -.:"v : • 

This report presents "preliminary .findings from Phase I of "A Demonstration and v 
Evaluation of Quality Assurance Procedures for the School Nutrition Programs." • 
The project's mandate (contained in PL 97-35) is to develop and validate methods of 
preventing and detecting, the misreporting^of income and family -size information on. 
school meal benefit applications. The project is being conducted in two phases 
corresponding to jthe 1981-82 and 1982-83 sdhiSol years. . Each phase will test .a 
different set of quality assurance procedures. Phase 'I of; the project involves an 
experiment^carriecfout in 13 School Food Authorities' (SF As) designed to (1) examine ^ 
the effects of changes in the school meal application process mandated by PL 97-35, 
(2) assess the impact of two low-cost quality assurance procedures, and (3) discover 
reasons for. applicant misreporting. Pnto II, being conducted during )he 1982-83 ( 
school year, will test more complex ^gonte documentation anc^ver if ication 
procedures on a large sample of SF As. 



Analysis of the Phase I data has, to this point, been directed/ toward providing 
preliminary indications of the impact or^ongressionally-mandated changes in the 
school meal apTSlication process and testing two quality assurance procedures. This, 
is one of several reports on Phase* I findings. A subsequent report will present 
findings from in-home audit interviews that were conducted in the spring of 1982 on 
a sample of program participants in the Phase I SFAs. The in-home audits assessed 
the completeness and veracity of . income ancf family size information reported on 
applications, examined reasons for misreporting, and developed and validated a 
model that predicts characteristics^ applicants who misreport (an "error-prone 
model"). The model ' is simUar to models used by the IRS and other agencies for 
targeting audits and case reviews. ; - t * 

Phase I uses an experimental design which allows analysis of the effects within 1, 
participating - SFAs of the quality assurance procedures' and changes in the. 
'application process^ However, because the participating SFAs were not selected in 
a manner that assures that the sample is representative of all SFAs in the nation, no 
strong inferences can be made concerning the national.impact of cfianges mandated 
by PL 97-35 or the two low-cost quality assurance^procedures. > 



Findings Associated with Application Changes ' ,. ; .;■ , 

> Three .Con^ application propedure wei^ 

reviewed: removal of, the free meal benefit guidelines from'iappUcatiori materials; a ■ 
new appUcatijbn form that requires a list of all adult* household members with Socia^ 
Security numbers .and a detailed breakdown o4lncome sources; and the elimination 
of hardship income deductions. v 

.'/ A 1980/ study conducted by the USD A Inspector General's Office, which has 
responsibility for delecting and preventing fraud and abuse in FNS programs, 
concluded 'that a portion .of applicants adjust their reported income on" the basis of • 
eligibility Sidelines to assure themselves of meal benefits. An important change in. 
•the meal benefit application process mandated by PL 97-35 was that the application 
materials! sept to parents include the reduced-price income eligibility guidelines 
orily, and not the 'free -meal guidelines. Removal of free-meal benefit guidelines 
from .application materials resulted in no discernible change in the percentage of , 
program/ eUgibles reporting incomes on or just under the free-meal eligibility cutoff^ 

; point. L school yearfl980r8i, when free-meal benefit guidelines were included, 2.2. 
percent-.o f free-meal;: benefit recipients reported annual incomes within $200 or less , 
6f the 1 free-meal ' eligibility cu^f point. For school y^ar 1981-82, when the 

- free-meal guidelines were removed from 4he application materials, the figure • 
remained constant at 2.2 percent. 

the new model. application If or m foi? school year 1981-82, which requires Social 
Security numbers of aA adults in the household arid income by source, was shown to 
.'have a variety. of effects oh applicant income 1 ^ reporting, program participation, and 
program costs to the Federal government. Program reapplicants using the new 
application form reported an average annual increase in income more than $500 
. greater than program, reapplicants using the prior-year (1980-81) model USDA 
..application form; This finding strongly suggests that the new application form has 
v beeri at least partially successful in preventing income from beinj^nder-reported on 
• meal benefit applications. * * 
./'""'• The number of eligibles approved to receive meal benefits was approximately 
40 percent lowerm schools u§ing the new appUcation form than in schools using the 
!pr^^^/for»/^ft6larlyi reduced-price;'' lunch eligibility was approximately 15 
i pe^aenjt lower in new application schools than in prior-year appUcation schools.!/ 
I ,T$4 reduction in i program eligibility associated with the new application form 
/ translates into lower program costs for 'the Federal government. In schools using 
new, appUcation form, Federal lunch subsidies averaged 52.5 cents per student 
per day. '• V : " , ' . ■ ■ v. ' , • ; ■ ' /; 




■4 



In contrast, the average daily per student slulwldies were 56,8 cents for schools using 
the prior-year application form,£/ , ' ' 1 • 

: Another Congresslonally-mandated change in the application .procedure was the 
repoyal.of allowable income deductions for special/ hardships when determining 
m eligibility,- The special hardship; deduction was removed in mid-school year 
81. following requirements of PL 96*499. The hardship income deductions were 
'found feaffeot the eligibility status of less than 2 percent of program, recipients for 
school year ,1980-81 -in the Phase I SF'As. Associated with elimination of the 
hardship deductions in school year 1981-82 was ^slight decrease in overall number 
. of program eligibies' and a slight increase in the percentage of recipients reporting 

incomes on or near the reduced-price eligibility guideline. • 

< . ■. ■ < ... •• • ■ 

• • » • ■ .•■ '• >• : . • • -: • • . • . . . . 

Findings" Associated with the Quality Assurance Procedures 
. The two quality assurance procedures were (1) a notice accompanying program 
^benefit applications warning that a sample of applicants, would be audited, and @) a 
notice accompanying -program benefit applications specifying ip detail the sources of 

-^ flC om^to^report-and-how-to-determine-household size.1 — ■ — ~~~ — — . 

The quality assurance treatments, used in association^ith the new applications, 
" had the same impact as the new application alone. . However, they had no 
{discoverable additional effect on applicant reporting behavior, program 
"participation, or program costs in Phase I SF As. . 

One of the Phase I sites, San Diego, did not adopt /he .new application form. As 
a' result, the quality assurance procedures . were used in conjunction with the 
prior-year form that .did not/equire social security numbers or itemized income. In 
San Diego the qiamy a^sU^nce procediu'es (used in conjunction with W prior-year 
•application) were associated' with 'lower " rates of program eligibility and lower 
Federal prbgram costs. One ^hypothesis that may explain this finding is that the neW 
application form 'is a stronger procedure than either of the quality ' assurance 
notices. Therefore, the notices, may be, effective When used with the prior-year 
' appUcation form, bu^have no additive effect when used with the new form. v 

Conclusions • v 

While the, new application form appears to have been at least partially 
successful^ preventing income under-reporting and presenting ineligible individuals 
from Staining program benefits^two major issues have not been addressed fully. 



Findings to date do not distinguish between reductions in eligibility due to the' 
prevention o{ fraud and abuse end reductions due to the new application creating 
barriers to participation by eligible individuals; nor do the' findings determine how 
much of &e problem of applicant mlsrtporting remains, 

The ; experience gained from conducting Phase I -is being -used to improve the 
design and execution of- Phase II; witjh spealdl emphasls'on testing. a variety of 
quality assurance measures, including,* requiring documentation with the application 
and foUow-up verification. procedures. Phase if will go beyond Phase I -in several . 
Ways* Phase H will test and refine error-prone models of applicant misrepdrting to 
.permit the cost -effective targeting of application verification efforts. Phase II wilK 
MjtfnduCt cost -benefit "analyses of quality assurance procedures and assess the burden 
'?f!|lse procedures may place on. states and local SF As. Finally, Phase II will 
dbt^n^h. between reductions in program participation due 'to a procedure 
preventing fraJd and abuse and redu||ions in program participation due to a . 
. /procedure creating barriers^to participation by e^gible indw .< 
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NOTES y\ ■ V 

.' . . • . > ■ .■ y j. -. - > ' i 

1/ An "eligible" student is defined asa' student who has a current, completed and 
approvedMearWnefit application on file. V • i 

2/ rAferage daii# student lunch subsidy was defined as the total Federal lunch 
~ . pfjjgxarn subsidy f pr a month divided both by the number of days in the month 
-OTiS vfl^f served and by^the'average daily attendance for the month. 
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, OVERVIEW \ 

* I \ " ; I 

This report presents f indings from preliminary analysis addressing .the impaot of 
Co*ngressioQally-mandated changes in the school meal benefit application process 
and of . the. low-cost quality assurance procedures implemented in Phase I of .the i 
Income Verification Pilot Project (IV PP). The presentation Hs based on an analysis 
of two of the three data, sources being 1 used to evaluate the Phase I quality assurance • 
procedures: meal benefit program eligibility and participation records and abstracts 
of meal b enefit' a p plications. Late r analyses will enrich and extend the- evaluation ! 
using' in-home auditodata obtained from applicants. The report divided into six 
chapters: ■ , t ' y ■ / ' 

• A review, of the background and current status of Phase \(E of the 1,VPP 
project; • ' \ 

• An analysis of the effects of 'CongressionaUy-mandated\appUcatiori * 
changes and experimental quality assurance procedures, on. program 
eligibility and program costs to the Federal govesnmenti , \ . 

• An analysis* of the effects of Congressionally* mandated application 
; changes and experimental quality assurance procedures on meal\benefi,t 

applications; ,.' f \ . 

• A summary of findings to date; . • . \ 

• A discussion of the implications of the finds to date; and 

• \ Appendices on the statistical methods used in estimating effects, and 

sample applications. ; 



BACKGROUND AND CURRENT STATUS OF 
PHASE; I OF. THE SNAP# PROJECT* 



FNS implemented two inexpensive low-burden quality assurance procedures for 
the free and reduced-price school meal application process on a trial basis during ^ 
the 1981-82 school year. These procedures- were; (l 1 ) a written warning- to 
applicants that they might be audited, and (2) a guideline that identified income 
sources to be listed and described how family size should be reported on the 
application. In July of 1981, 13 school food authorities (SFAs) -were recruited to 
participate as pilot sjtes in. Phase I, Exhibit 2.1 lists the SFAs that volunteered to 
participate in Phase I of the project. Shortly thereafter (in August), PL 97-35 was 
enacted by Congress. PL 97-35 introduced significant Changes in the application 
procedure for free and reducedrprice schoad meals. USDA responded to PL 97-35 by 
introducing a new recommended application form shown in the Appendix B. This 
new application differed from previous applications in that it required the applicant - 
to list-total-monthly-f amily-income_by_source^(rather .'than total income only) ; and 
required the names and Social Security numbers of . all adults. Although thgre was 
^>ot sufficient time to publish regulations before the 1981-1982 school year, states 
respilBS||by modifying existing application forms. 

V : Th^hase T- study design was modified to include these legislated application v 
changes. The result was the creation of as many as four different applications to be 
used by the Phase I pilot SFAs: * 

(1) 1981-82 Schoo l Year Application "only (Treatment. Group 1). This 
application was similar to applications used by most SFAs throughout the 
country for the 1981-82 school year. It requires' that the applicant list 
monthly family income by 'source and the Social Security number of each 
adult. family member and provide other information needed to process the 
application. - ' •"■ 

• • ^ t: 10 . ■ ?• • 
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(3) laai^ aa aohool Yoar .ApBligaiten and Aid it Warning treatment Groop 3 J. 

thteniitoit^irHittilftil 'to the epeUaaiien $mtpm t ahoy* in 

mttition, a warning was etteehed advisinj( that a. sample o( eppUeents' 
« ' ■ would be audited, ™ is toe first of the meijperaivft .iw^urOen mW 
' assuranse procedures pilot tested by the study., (gKhi&it%8 is a eepy or 
the warning notiee,) * 



described in ,UL above, In addition, guidelines were attaehed that 
specified' income sources to fie ineluded on the application and the 
definition of the family siae* This is the second inexpensive tew^urden - 
quality sssuranee. procedure pilot tested in the study, tphroit '4,3 is a 
eopy of the guideline notiee.) ■• ^ 

(4) t98(Mt School Year Application (Treatment tlroup 4), The application 
^elt hylf^niVent groiip 4 dirfiTritl from that used by treatment groups I, 
L.and 3 in that it did not requlr# Sooiat 8eaurity numbers and an itemized 
> breakdown of income, Tills appiioation was identical to the appiioation 
• used in the 10«t>8l school year »with two exceptional the hardahip 
v deduction was removed and on Ly redu«<i«prioe eligibility guidelines were 
provided with the applications, (In past yeara applicants were provided 
with both free and reduced-price eligtblUty'gutdellnes,) 

- The applications were then distributed and used in the fail of the 1.991*82 uohooi 
year In a sample of schools in each of ,the Phase . I SFAs. In ome of the M 8FAa that 
process applications manually (non-computei'teed sites), all four applications were 
used In o sample of elementary schools. Two elementary schools were assigned at 
random to each of the four treatment groups, -• 

Four of the 13 SFAs use computers In some aspect of tho froo and 
rcduccd-prico school meal application/certification process. Computerized form 
processing procedures made It necessary for these four districts to use only tho 
1981-82 sohool year applications. (The use 'of divergent application forms would 
have impodod their certification process.) Therefore ail schools in tho computerized 
districts wore assigned to one of the three groups that used the 1981-82 
applications. As a result, the 1981-82 Application only, the 1981-82 
Application/Warning, or the 1981-82 Application Guideline were distributed to $li 
elementary and secondary students. / 1 

Due to real weJrld constraints, it was not possible to implement the treatments 
uniformly in all Phase I SFAs. The treatments' as implemented in the computerized^. 
SFAs varied significantly from the non-computerized SFAs. In the computerized 
SFAs all schools in the district participated in the project; in the non-computerized - 
SFAs, eight elementary schools per district were selected to participate. One 
computerized SFA, San Diego,, introduced several changes in the study procedures. 




. ■ w ■ : ■ • ■ 

EXHIBIT 2.2: ; WARNING, NOTICE 



*. NOTICE TO APPLICANTS 

PLEASE FILL" .OUT THE ATTACHED APPLICATION FORM 
VERY CAREFULLY. DURING THE COMING SCHOOL YEAR WE 
WILL CHECK SOME OF THE APPLICATIONS. YOUR 
APPLICATION MAY BE REVIEWED. WE WILL CHECK THE 
• CORRECTNESS OF FAMILY INCOME AND . FAMILY SIZE 
INFORMATION THAT WAS REPORTED ON THE APPLICATION. 
YOU MAY BE CALLED FOR A HOME INTERVIEW TO REVIEW 

YQUR APPLICATION ." / ' „ • f * 

' — ~ y 
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EXHIBIT 2.3: gUIDELINE NOTICE 



. ; NOTICE TO APPLICANTS 

THE ATTACHED APPLICATION FORM ASKS tfOU TO REPORT VoUR FAMILY 
INCOME AND THE NUMBER OF PERSONS IN YOUR FAMILY. WHEN YOU 
REPORT THIS INFORMATION, PLEASE PAY CAREFUL ATTENTION TO THESE 
HELPFUL GUIDELINES: 

FAMILY INCOME 

YOU MUST REPORT income .for you ftrtd all other members of your family. 
Income from the following sources should b<e> included: 



•5- 



• unemployment 
compensation 

• Supplemental 
Security Income 
(SSI) or Social 
Security 
Survivor's 
Benefits . 

• alimony or 
child support 
payments 



• wages, salaries, 
tips, commissions, 
or income from 
self -employment 

t net farm incorrfe 

• ■ pensions, annu ities, 

or other retirement 
income including 
Social Security 
x retirement benefits 

t public assistance 
and welfare pay- -i 
ments (AFDC) 



DO NOT REPORT: 

t scholarships or other educational benefits 
t food stamps 
FAMILY SIZE 



• ' disability benefits, 

j • including workmen's 
compensation 

• veteraiVs subsistence 
benefits 

• interest or dividend 
§ income 

• cash withdrawn 
from savings, 
investments, trusts 
and Qther resources, 
which would be 
available to pay 

for a child's meals 

• other cash income 



You should count in your family any person, related or unrelated, who lives in 
your household and who shares household expenses or meals. 

IF YOU HAVE ANY QUESTIONS, PLEASE CALL YOUR CHILD'S 
SCHOOL PRINCIPAL. ' 



San Diego did not approve the use of in-home audits and used a noncomparabie 
application form, that does not require Social Security numbers, a listing of adult 
household members, 0/ detailed income sources. Another computerized SFA, 
Dayton, assigned schools to treatment groups in a nonrandom fashion so that two 
treatment groups* consisted almost exclusively of public elementary schools while 
the third treatment group consisted of public high schools, private religious schools, t 
and special education public schools. Dayton also modified study procedures by 
using an application requiring employer name aq^address, a second application with 
an audit warning notice, and a third application containing income and household 
size guidelines. 

Phase I SF As volunteered to participate in, the study and were not selected 
randomly. Therefore, the sample is not necessarily representative of all ST As in the 
nation. No strong inferences concerning the national impact of the new application 
form on the school meal program can be made from Phase I of the IV PP project. 

/ To test the effects of the new application form and the quality assurance 
' procedures on the participating SFAs, three data collection efforts were planned. 
First, for participating schools in the nine non-computerized SFAs, meal benefit 
applications for the 1980-81 and 1981-82^ school years were, abstracted and matched 
across the two school years. For all applications, date of application, income by 
source, and reported family size were recorded. In total, over 35,000 applications 
were abstracted. Once abstracted, the data were keyed, verified, structured into a 
common form across SFAs, aggregated to the school level, and converted to a 
computer analysis file. The application data allows an assessment of the impact of 
the quality assurance treatments on who applies for benefits and what they --.report 
on their appUcations. Analysis, of the appUcation abstract data is reported in 
Section 4 of this paper. 

The second data collection effort involved meal befle fits program eligibility and 
participation records. To test the effects of the new application and the quality 
assurance procedures on aggregate program participation, the following data were 
collected for the months of November 1980 and November .Igl from all 
participating schoo^h the 13 Phase I SFAs: . ( 

number of complete operating days for breakfast and lunch service; 
number of full price, reduced-price, and free breakfasts served; 
number of complete full price, reduced-price, and free lunches served; 
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' . • average daily attendance or average daily membership; 

• total number of students certified for free meal benefits and total-number 
of students certified for reduced-price meal benefits. 

Once collected, the data were keyed, verified, ana converted into a computer 
analysis file. The analysis was- conducted in two stages: varitble and model 
specification and testing; and model computation and interpretation. Sections 4 and 
5 present the results of these analyses. 

The third data collection effort is in-home audits with a sample of program 
participants in participating.SFAs. The audits are currently being analyzed with an 
expected completion date in December, i982. The in-home audits will assess the 
completeness and veracity of income and family size information reported on 
applications, examine reasons for misreporting, and develop and validate a model 
that predicts characteristics of applicants who misreport (an »error-pron| model"). 
Findings from the in-home audits will be the subject of a.later report. 4 
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EFFECTS OF THE NE& APPLICATION F^ORM, AND EXPERIMENTAL 
QUALITY ASSURANCE PROCEDURES ON PROGRAM ELIGIBILITY, 
- PARTICIPATION, AND COSTS T<^THE FEDERAL GOVERNMENT 



This section describes the preliminary findings of an analysis^ Phase I SFA - 
program eligibility and participation records. "There are three subsecftonsj__>- 

■r ' '■ 

3.1 The conceptual model used to analyze 'the program eligibility and 
participation data; t . 

3.2 The ^effects . of the new application form and experimental quality- 
assurance procedures;on program eligibility and participation; and 

3.3 The effects of the new application form and experimental quality 
assurance prflsedures qn program costs to the Federal government. 

3.1 The Conceptual Model ' 7 / 

The -purpose of the school meal application process is' to make, programs- 
benefits accessible to eligible individuals and to deny access to ineligible ■ 
individuals. The Phase I quality assurance- treatments andCon^ressionally-manda ted '•, 
application changes were designed to deter applicants .from misreporting " 
information used tp determine program eligibility. If the procedures are effective, 
they can be expected to reduce the total number , of eligibles certified for the . 
program and program participants. In fact/ the Office of the Inspector General v 
estimated that in school year 1979-80, 28.j) percent v of all . approved free and 
reduced-price school meal applications werV sufficiently in error, to alter true, 
eligibility. If this estimate is correct, the potential for reductions in program 

participation and Federal costs are considerable. 

t ■ • ■ . 

Although an effective quality assurance procedure wUl reduce total program 

participation, a reduction in program participation associated with a quality 

assurance procedure does not necessarily prove that the procedure has deterred 

ineligible applicants from applying or misreporting' their eligibility status. The 

procedure may instead act as a barrier to eligible, individuals-a barrier thKy 

prevents them from applying or motivates them to under-report their true eligibility 
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status, when applying. Both the barrier and deterrence, effects 
procedures reduce overall progra^ participation. That is, reduction in, F9^ m 
participation associated with, a quality assurance procedure is the sum of barrier and 
deterrence effects.^/ Because of this.^the inferences possible about effectiveness 
of the Pha%e 1 experimental quality assurance procedures on the basis of aggregate 
program participation data are asymmetrical— no reduction in program * 
participation is strong^evidence of treatment ineffectiveness, .'whereas a reduction in 
program participation is not direct evidence of significant deterrence effects.2/ ^ 
V In dbritrast, aggregate program participation data does provide- strong direct 
evidence of cost savings associated with the quality assurance .procedures. Because 
the Federal government reimburses SF As a fixed amount by benefit category per 
meal served,,.' any "'.reduction ' .in" number • of -.meals _ served has an associated cost 
savings, -Although it is desirable to maximize savings from deterrence and minimize 
savings from barriers, total savings associated with a treatment derive fro m> the sum 
of'Stier and deterrence effects arid are unaffected by the relative contribution of 
barriered deterrence effects. As a result, aggregate program participation data 
can be used to estimate cost savings arising from the experimental trea^nt and 
hew application form. ■.• * 
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3.2 Effects of the New AppUcatiori Form and Experimental Qua lity Assurance 
Procedures on Program Participation r " v A . 



Effects of the new application form aW quality assurance procedures wer;e 
estimated through tests of three a priori hyp&nesesj, -j- ■ ^ , <■■ 

1. Program participation "in the schools employing the new application form 
' is lower than in? schools employing prior-year forms. g 
' 2. Program participation in schools using both the new application form with* 
■"' either the warning notice or the income guideline is lower than In schools 
* v 'using only the*new application form. • .'.•„•'!',.. • 

3. Program participation for schools using the warning notice is. unequal to 
participation in schools using the income guideline notice. _ .«? 

The first hypothesis 1 tests whether th> new . gfe^ on form V f< f ted .program 
participation. The second hypothesis tests Whet?|Ftte' tocofoe guideW^otice or. 
warning notice had any additional effect on participation' dver and beyond effects of 
the new application form. The third hypothesis tests whether one or the other of 
the two notices had a greater effect on program participation. 

* * * 

All three hypotheses were independently tested to determine treatment effects 
in the nine non-computerized Phase I SF As for each of six participation categories 
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/ in Member 1981>^fr^e ludch, redu<Sed-|Rce lunch,' -fiill price lunch, 'free breakfast, 
* ie||ced^ricel breakfaS, : and full price^break fast. Rate of program participation 
v|^'lfefihed;for the Income"v er if ication Pilot Project as. the proportion. of average 
d|il^.ienrpUment receiving lunch or 'breakfast benefits. It should be noted that this 
de&itiort of . program participation differs from others used by previous research. 
.Mrlptanec, program participation has been defined in the. National Evaluation of 
ftne^crtool Nutrition Programs as' the percent of e'Ugibles ipgardlesaof whether they 
|j:ha^e ! an appUcatioh on^file.' • « i 

v f/|S^ The most substantively and ;statisti'caUy., '^^4ufe|p^,' of "° the \D$ W , 
~ ||Slrcatibn'form and experimental treatments was^i^BStobe on participation 
v-ilOT'ftee-luhch benefits. This was not' unexpected because^ lunches represent the' 
-ll^igest of the six benefit categories and there f ore, is V $ie eatery in which 
/^'•Ix ueatm^nt effects could be most easily detected. •'■». J 

, As noted above,-* three hypotheses were tested. |irst, participation data ' fqj. 
schools using the old, application form were contrasted, with participation data for 
vschools using the new appUcation form. - This contrasf allowed es^mation of the 
^effects' of adding Social Security numbers, income sources, and^a listing.'jOf adult 
. househol^memtrers to the application ' ■for mi Once adjustments were made for the 
rat/of program" participation in the prior year, the average free lunch participation 
rate was approximately 38 pereenft for schools employing tile prior-year .application 
form and 34 percent for schools employing the hew application form.3/ Stated 
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differently in the Phase I noncomputerized sites, the" new appUcation form was 
, associated with a reduction- of approximately 10 percent in the number of free 
lunches served (10-0 f(l - 34/ 38)). 4/. (A difference of this magnitude has a •' 
probability of having' occurred by Chance of less than one in a thousand., We can 
represent this 'fact -by the equation p<.0Gfl.) This stage of the, analysis found" no 
h- significant interaction between SFA and "effect of appUcation ^form.0 That is, the 
impact of the new appUcation form was nearly constant' across. SF As except for 
random .sampling variance. " •;' \ 

Tests of the second and, third' hypotheses could uncover no, additional impact of * 
' the quality assurance procedures on program participation in free lunches beyond, 
.'the effects ctf the new appUcation form. Exhibit 34, summarizes the findings. As ^ 

can be seen, all three experimental jpowsMpg the new appUcation form had lower 
' than average ctfily percentages of students receiving free lunches than did schools , 
using the prior-year appUcation form. However, the groups using the experimental 
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EXHIBIT 3.1r FREE LUNCH PARTICIPATION IN 

s BY TREATMENT GROUP* 
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qufiy assurance procedures lfeve only minor differences in participation rate from 
|3|group using only the heyT^Ucation form. Moreover, the direction of the; 
difference in participation rate is the opposite ' of that hypothesized; theNj 
.participation rates were higher' i&the quality assurance treatment groups. 

. Identical 5 analyses were cohdujrted for the remaining five program benefit 
categoriesi -reduced-price lunch, full price lunch,^ free breakfast, reduced-price 
breakfast, T and/ fuU price breakfast. In .only one case were .any of the thre^e. 
hypotheses confirmed at a level of statistical significance of .05. The one case was 
program participation in .reduced-price lunch.es. . After adjusting for 1980 program 
participation, an average. reduced-price lunch participation rate of 6.8 percent was 
found for schools using the prior-year application; form and a 5.8 percent 
participation rate for schools using the -new application for m (p<.05). As was th 
case in free lunch participation analysis, no additional effect was discovered for t 
experimental quality assurance procedures. Nonsignificant treatment differenc 
were, found fpr the remaining four categories qlFfrogra m participation: full price N 
lunch, free breakfast, reduced-price breakfast, and full price breakfast. 

It is not surpising that no treatment effects were discovered in the school 
breakfast program, y Less \han half of the \ Phase I schools are in the breakfast 
program and in those schools breakfast program participation' averages 25 percept. 
*-less than lunch program participation. As a result, treatment effects m the 
'breakfast program would have had to have been much larger than those found in the 
lunch program to be detectable by the Phase I experimental design. ^Therefore it 
woul^be false to conclude that .the new application affected participation in the 
lunch program and not the' breakfast program. . The most that 'can be concluded is 
that the effects were, only detectable for the lunch program. 

The analysis of participation data conducted for the nine .non-computerized 
"" SFAs was independently replicated' in two of the four computerized SFAs.5/ In 
Akron, the one computerize/ SFA that fully implemented the new. application form 
and the two quality assurance procedures, no differential jsffects of the procedures 
on program participation/Were discoverable. Therefore the Akron analysis repeated 
the findings from the nine non-compuferized SFAs. (It is important to remember 
that group 4-prior-year form— was not implemented in the computerized SFAs, 
therefore comparisons of new form and prior-year form schools could not be 
conducted.) However, in- San tfiego, significant differences were discovered .in 
program participation rates across the treatment groups. * » 
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As noted above, San Diego did not use the new Federally recommended 
application form in any of the 'three 'treatment /groups. The three treatments as 
implemented in Sart Diego are: ' . , • 

(A) 198a-81 School Year Application (Treatment Group 1 A). This application 
required only total monthly income and total number - of household 
members. It 'did not require a listing of family income by source, the 

•. \Sociar Security numb~er of each: adult family member, or a listing of all 
adult family members. • 

(B) 1980-82 School Year Application and Audit Warning (Treatment Group B). 
This application is identical to the application described above. In 

• addition, a warning was attached- advising that a sample of applicants 
• Would be contacted by telephone to verify reporting income information. 

(C) 1980-81 School Year Application' and Income/Household Size Guideline 
(Treatment Group C). Again the application is identical to that describee 
in (A) above. In addition, a guideline was "attached specifying in detail 
income sources to report and the definition of household^size. 

In summary, San Diego implemented the quality assurance procedures in 
conjunction with the prior -year,,appli cation form rather than in conjunction with the 
new Federally recommended form. Therefore San Diego created an opportunity to 
test the effects of the quality assurance treatments as separated from, the new 
application. . 8 

Analysis of San Diego school lunch program participation records showed a 
significantly lower rate of participation in free lunches' in treatment groups B and p 
than treatment group A (pn<\0i). Exhibit 3.2 graphically presents this finding. As 
can be seen, the warning notice and the income guideline notice appear to have had 
equal and substantial effects on free lunch participation. Average /free* lunch 
participation in schools using only the prior-year form (group A) was approximately 
3 ; 0 percent, whereas average participation for schools using either the warning or 
income guideline notice was slightly under 24 percent. No interaction was found 
between school type-elementary, junior high, high school, or special education— and 
treatment group. e.ffect. That is, the effects of the warning and guideline Notices 
were nearly identical for the four types of schools except for small random 
differences.^/ • 

Thb finding raises the question of why the experimental quality assurance 
procedures aff ected program participation- in San Diego but not in any of the other 
Phase I SFAs. One possible explanation is that the new Federally recommended 
application form that requires income sources and Social Security numbers is a 
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stronger treatment .than the experimental, qualify assurance notices and therefore < 
the notices are effective when used with the old application form but have no 
additive effect when used with the new form. At this time, available evidence is 
insufficient to regard this explanation as anything more than a working hypothesis. 

3.3 Effects of the New .AppUcation Form and Experimental Quality Assurance. 
Procedures on Program Costs to the Federal Government ~ 

The Federal government reimburses SF As fixed amounts per meal served by 
program category. For example, the Federal subsidy for reduced-price breakfasts is 
currently 28.5 cents. Because the Federal government subsidizes individual school 
, meals at a fixed rate, reductions in prograrn participation translate directly into 
cost savings for the Federal government.7/ • 

The last stage of our analysis of the Phase I program participation records , 
translated participation data into subsidy costs for the Federal, government. The 
basic unit of the analysis was "average program costs per student per.day."B/ 
Exhibit 3.3 presents the results. The Federal government spent, an average of 56.8 
cents per student day for the lunch program in schools using the priori-year 
application form versus an average of 52.5 cents per day for schools using the new 
application form (p < .0005). Although there were small differences in average jjlaily 
\ costs among the three groups using the new application form, these differences were 
statistically insignificant and in the opposite direction than expected. (The 
experimental quality assurance procedures were expected to decrease, not increase 
program costs.) N<) meaningful differences could be found by contrasting the four 
groups in terms of average daily costs of the breakfast program (See Appendix A.). 



\ In the computerized SFAs, findings from the analysis of program costs 
paralleled the findings f$r program participation. In Akron, rib effect of the quality, 
assurance procedures was jjiscovered. In San Diego, average daily lunch program 
costs for schools using either the warning notice or the income guidelines were 
lower than in schools using only the prior-year application form (p<.05). Average 
daily lunch program subsidies were 47.5 cents for schools in Group A (1980/81 
. application form), 41.1 cents for Group B (1980-81 application form with warning 
notice), and 41.4 cents* for Group C (1980-81 application form with income 
guidelines). Therefore in San Diego the' quality assurance procedures were 
associated with program cost savings. As this finding is limited to a single SFA, 
generalization is not possible. Exhibit 3.4 shows these results. 
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EXHIBIT 3.3: 
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EXHIBIT 3.4: AVERAGE DAILY LUNCH PROGRAM FEDERAL 

REIMBURSEMENTS PER STUDENT BY 
TREATMENT GROUP IN SAN DIEGO 
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NOTES 



V Changes in rates of progpram participation can, of course, result from many 
1 different sources, including changes in eligibility guidelines, price changes, 

demographic changed, and so on. The great advantage of the experimental 
design used in the Phase I evaluation is that it allows identification of the 
unique effects of experimental quality assurance procedures employed: in the , 
treatment groups. That % the experimental model permits the evaluation to 
account for changes that might naturally affect program participation. 
! Therefore in this report, reference to the,; impact of quality assurance 

procedures oh program participation are to ttie experimentally determined 
unique effects over and beyond any impact of prices, guideline changes, etc. 

2/ in the abstract, a second possible interpretation exists. Deterrence effects 
may be positive and barrier effects negative and sum to little or no* impact on 
total program participation. This outcome is very unlikely on both theoretical 
and empirical grounds, however. Empirically, such "suppressor" effects are 
very rare in quality assurance research. Barrier and deterrence effects are 
generally strong and positively correlated. In this particular case there is 
nothing in the experimental treatments that could be reasonably expected to 
increase the application rates of eligible individuals' and thereby^ result in 
negative barrier effects. 

3/ Free lunch participation rate for a school was operationalized as FAExD) 
~ ' ' where F is the total number of free lunches served in November 1981, E is the 
.total enrollment, and D is number of days lunch was served.. 

! -4/ ' .Again, it should be noted that this difference is over arid beyc>nd any effects on 

' participation caused by other factors such as price and guideline changes. 

* * 5$ ; ' Results for two computerized SF As have not been presented. In one SFA, the 
number of schools was too small. In the second SFA, schools were not assigned 
to treatment^ randomly. 

I ■ y v Analysis of experimental treatment effects in San Diego was limited to a 
review of program- participation records. /Applications were not abstracted in 
the computerized SF As. — 

7/ Average Federal reimbursements (including Federal lunch commodity 
donations valued at ll<t per meal) for the period September!, 1981 to June 30, 
1982 were: Free lunch— $1.2025, Reduced-price lunch— $0.8025, Full-price 
lunch— $0,215, Free breakfast— $0.57, Reduced-price breakfast— $0,285 and 
Full-price breakfast-$0.0825. 

i . i 
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- NOTES (Continued) 

Averaee daily Federal reimbursements per participant were separately defined 
for the lunch and breakfast programs. For the -lunch program, average daily 
Federal reimbursements for a given school were operationalized as: 

(F x 1.2025 + R x .8025 + P x .215)/(E x D) 
. where: - 

F is the total number of free lunches served in November 1981; . «- 
R is the total number of reduced-price lunches served in November 1981; 
P is the total number of full price (paid) lunches served in November 1981 ; 
E is total enrollment; and . ' 

I> is number of days lunch was served. 

Average daily costs of the breakfast program were defined as: . 

(F x .57 + R x .285 + P x .0825)AE x D) 
where F, R, P, and D were defined so as to refer to the breafast program. 
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EFFECTS OF THE EXPERIMENTAL QUALITY ASSURANCE TREATMENTS 
AND PLi97-35 MANDATED CHANGES ON MEAL BENEFIT .APPLICATIONS 

.This section presents- the findings from analyses of the meal benefit 
applications in the nine non-computerized Phase I SFAs. The analysis addresses four 
topics: ' • • '..7_"'~:'': .': \- 

/4.1" The effects of removing free meal income eligibUity guidelines from 
applications materials sent to parents; 

4.2 The effects of the new application form; . 

4.3 The effects of eliminating the hardship income deduction; 

' 4.4 The effects of the experimental quality assurance procedures. 

4.1 Elimination of Free Meal Eligibility Guidelines from Application / 
Materials" " , . .. ■ ■ ~~ : ■ " • - 

An important change in the meal benefit application process mandated by PL 
97-35 was that the application materials sent to parent^ include the reduced-price 
income, eligibility guidelines only and not the free meal guidelines. This change was 
made following a nationwide study conducted by the USDA Inspector General in 
1980 to investigate allegations c that a number of applicants refer to .income 
eligibility guidelines and adjust their reported incomes to assure themselves of meal 
benefits. The study concluded that in approximately 3 percent - of the 5,000 
applications examined, households may have used the income, guidelines provided 
with the application form to derive their reported family income. ' , . 

' As a preliminary and partial test of the effects of the elimination of the free 
meal guidelines from application materials, a comparison was made of the 
percentage of free meal benefit eligibles who reported an annual income within $200 
of the free meal' eligibility cutoff point in school year 1980-81 (when free and 
reduced price meal guidelines were supplied with applications) With the percentage 
of free meal eligibles who reported an annual income within $200 of the free meal 



eligibility cutoff point in school year 1981-82 (when free meal guidelines were not ; 
supplied with appUcations).l/ For school year 1980-81, 2.2 percent of free meal 
.benefit eligibles reported an annual income on or less than. $200 under the free meal 

' eligibility cutoff point, For school year 1981-82, the figure remained constant at. 

5' 2.2 percent, therefore the analysis was unable to detect any effect of removal of 
the free meal guidelines on the proportion of applicants who reported incomes on or 
very^near the cutoff point in the free meal eligibility guideUnes.2/ ' 

4.2 E ffect of the New Application Form 

lh response to PL 97-35, the Department designed a new recommended meal 
benefit appUcation form. The new form differed from the previously recommended 
form in two substantive ways. The new model form: ±> 

■ • required the names and Social Security numbers of all adult household 
members; . 
• required a reporting-of income by sources. 

The changes in the appUcation form were motivated, in part, from results of a 
second study by the Office of the inspector General in 1981. This study concluded 
that 28.9 percent of a sample of meal benefit appUcations contained under-reported 
income, which resulted in students receiving meal benefits that they were not 
. entitled to receive. ... 

If the new appUcation form achieved its purpose of reducing the 
under-reporting of income, it is reasonable to, expect that school year 1980-81 
program eUgibles who reappUed for benefits in school year 1981-82 using the new 
appUcation form would report a higher average increase in income than those who. 
reappUed using the old appUcation for m.3/ 

To test this expectation, a comparison was made of the average change in 
reported income for the three Phase" I treatment groups using the new application 
form with average change in reported income for treatment group 4 that used the 
pr ior-yearr appUcation for m.4/ Exhibit 4.1 displays the results of this analysis and 
shows iwat, on the average, in schools where the new appUcation form was used, 
program reappUcants reported a mean increase in income of approximately $1,450. 
This Ltrasts with a mean increase of only $900 for schools using the prior-year 
appUcton form.5/ Therefore it appears that the new appUcation form has been 
at least\artially successful in preventing the under-reporting of income on meal 
benefit appUcations. 
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A comparison was also made of the three treatment groups using the new form ' 
with the fourth group that used the pr^year form, in terms of reapplication 
rate ,6/ percentage, .of applicants reporting incomes on or near the eligibility 
cutoff point, change in -reported family size, and percentage of total school 
enrollment certified to receive program benefits, .. , ' 

Average change in reported family size was compared across treat merit'gfo^ps ' 
to determine whether the new application prevented or reduced the over-reporting 
of the number of family members. The percentage, of applicants reporting incomes 
on or near c the fre'e and reduced-price eligibility cutoff points ' was compared to'j^ 
d'eter mine whether the^w application had deterred applicants from adjusting their ^ 
reported indomes on the basis of eligibility guidelines. To evaluate the effects of 4; 
the new application on, application and reappUcation rates, these two variables were ' 
compared across the treatment groups". None of these contrasts revealed any 
substantively^ statistically significant differences between schools using' the old 
application form and 'schools using the new application form. That is/ the new 
appUcation had no discoverable effects on . family size reporting, .program 
reapplication rates, "threshhold" income reporting, or total application rate. 

A word of caution is necessary here. The lack of findings should not be 
interpreted as indicating the new application form had no effect on reapplication 1 -. 
rate, reported family size, total application rate, or reporting of income near the '. 
eligibility threshhold. The most that can be reasonably inferred is that if the new 
application form did affect these variables, tie effect was not of a magnitude 
detectable by the experimental design employed in Phase I. Because of the small 
number of schools per treatment group (18) and the substantial random variance 
between schools on all variables of interest, the possibility must remain open that 
the new application form has generated important although undetected! effects on 
applicant behavior. , 

4.3 Elimination of Hardship income Deduction 

' Another Congressionally-mandated change in the application procedure was the 
removal oOllowable income deductions for special hardships when determining 
program eligibility. The hardship deduction was removed in mid-school year 1980-81 
following requirements of PL 96-499. School year 1980-81 data was composed of 
applications filed prior to this change. To assess the impact of the removal of the 
hardship deductions, a two-stage analysis was conducted. First, a measurement was 
taken of. the effect of hardship deductions on program eligibility and participation in 



1980 when the deductions were still allowable:. Second, an analysis was conducted of 
•the effects of the removal of the deductions on program participation and meal 
benefit applications in 1981. 

The hardship deduction affected the program eligibility of relatively few of 
meal benefit eligibles in the participating Phase I SF As in 1980. Only 1.6 percent of 
all free meil benefit edibles would have been ineligible for free meals if it were 
not for tjle hardship deductions. Similarly, only 1.5 percent of all reduced-price 
meal-benefit eligibles would have been ineligible without the hardship deductions. 

^Elimination of the hardship deduction appears to have resulted in a small 
reduction in overall program eligibility in 1981. This reduction was observed by a 
negative correlation between the percentage of program participants in. a school 
benefiting from the hardship deduction in 1980 and the average daily percentage of 
students receiving meal benefits in the school in 1981 (r =-.35, p<.005).7/ 

j There was also a statistically significant and positive relationship between the 
percentage of applicants in a school who benefited from the .hardship deduction in 
1980 and the percentage of applicants who in 1981 reported a change in income or 
family$ize that positively affected their eligibility status (r = .25, p<.05). One' 
possible interpretation of this finding is that the elimination of the hardship 
deducti6ni may have resulted in a number of individuals under-reporting their income 
in 1981 to retain program benefits. This interpretation is supported by the discovery 
of a positive correlation between the percentage of applicants in a school who 
- benefited from the hardship deduction in 1980 .and the percentage who, in 1981, 
reported an annual income on or within $200 of the yearly reduced^price eligibility 
guidelines (r = .28, p < .02). Therefore it would appear that some of the beneficiaries 
of the hardship deduction in 19-80 lowered their reported income in 1981 to, continue 
receiving program benefits.8/ 

4>4 Experimental Quality' Assurance Procedures 

The last step in the analysis of the Phase* I school application data was an 
analysis of the effects of the two experimental quality assurance procedures. Group 
1 .(new application form by itself), Group 2 (new form with income guidelines), and 
Group 3 (new form ,with an audit warning notice) were contrasted in terms of: 

I , • average change in reported income for program 0 reapphcants; 
\ • average change in reported family size for program reapplicants; 

. 33 0 ' 
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• . percentage 'M applicant? reporting incomes on -or near both, tl?e 
V - reduced-price and free meal eligibility cutoff point; , ^ / ■: ; 

' • application and reappli cation- rata ; | 

• average number of income sources reported on applications. f * • 

The first four contrasts j^Ueled the analysis of the effect^ of the new application 
form. In addition, a fiflf contrast .was conducted that compared the thrjee groups in 
terms of average number -of income sources reported to determine': whether, the 
experimental. quality assurance- procedures, 6Uaited a wider range of reported types 

of income. • > 

Where if was appropriate .to increase the precision of the contrasts/crior-year 
variables (such as 1980 average • income, application rate, and ■ percentage of 
applicants reporting incomes near the eligibility cuto?f<points) .pre held 
statistically constant across treatment groups. ! v v 

None of the contrasts revealed' any statistically or substl#iyelyj 
differences between the three groups' using the new application form." 
application data revealed no effects of the warning and guifa 
applicant reporting behavior.; Again, it should be ifcted that these fin*, 
indicate that the experimental quality assurance treatmentsj^d nofeffect on 
applicant behavior, but rather that if there were any effects they were not of a 
magnitude detectable by the Phase I experimental design. •/" 




' ',, • ■ ■ ; ; notes ■;. 

■M ' The operationalization of "threshhold"; income -.^^W^^^^J 1 -^ 
~ annual income of the eligibility cutoff point is based^njhe_>es 4 lts of prior 

- ' studies including those by USDA's .Office of Inspectdi^^»ex : A different 

operationalization would have produced a different perceffSte of applicants 

- who report "threshhold" incomes. V J - 

2/ The Dercentaffe of reduced-price meal benefit recipients who Sported annual 
" incomS Sn $200 of ' the cutoff point in the reduced-price guidelines 
. increased from 2.5 percent in school year 1980-81 to 4.7 percent in school year 
1981-82 (p<. 001). • • 

3/' Measuring, the effect of the «^ appUcatioh form on inco^ 

of reported Income change from prior year is superior to measuring the ettect 
in terms of absolufrtifferences in . reported income because program 

• : . eligibility requirements truncate the recipient- income distribution so that the 
mean income "of recipients who under-report their income cannot be assumed 
to be lower than that of recipients who report Aheir total income. 



NOTES (Continued) 
The statistical procedures used are described in Appendix A. 

The difference between the groups is statistically significant at the p = .2 
level, which means that there • is approximately a oneMn-five chance the 
difference is due to random sampling error, To rule out this possiblity will 
require further analysis, 

Reappli cation rate was operationalized as: ' 

wherfe R is the reapplication rate, B the' number of 1980-81 school year 
program recipients in school i who applied for benefits in school year 1981-82, 
and C is the total number of 1980-81 school year program recipients in school i. 

Because the correlation is based on aggregate school leyel data and not 
individual applicant level data,- its magnitude exceeds the expected individual 
level correlation (See L. I. Langbein and A. J. Liehtmah, Ecological Inference , 
Sage, BeVerly Hills/ 1978.). , ; 

The conclusion presented here must be viewed with caution because itis based 
on an "ecological inference." That is, conclusions about individual behavior 
Were based on aggregate school level data and not on direct observation of 
particular individual applicants who lowered their reported* income to maintain 
program benefits. To test for spuriousness at the aggregate level, controls 
were introduced for school year 1980-81 average income and percentage 
reporting inconi'e near the guideline. The relationship between school year 
1980-81 hardship deductions and both school year 1 1980-81 income change and 
threshhold reporting remained constant when school year 1980-81 average 
income and average threshhold reporting were held statistically constant. 
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. SUMMARY OF PHASE I FINDINGS 

Analysis .of the Phase I data has tq this point been directed at providing :. 
preliminary indications of the impact of Cpngressionally mandated changes in the 
school meal application process and testing two experimental quality assurance 
procedures. \ 

Three changes in the application procedure made by FNS in response to 
Congressional requirements were reviewed: removal of the free meal benefit * 
guidelines from application materials, a new application form that requires a listing 
of ail adult household members with Social Security numbers and a listing of income 
sources, and the elimination of hardship income deductions. 

Removal of free meal benefit guidelines from application materials resulted in 
no discernible change in the percentage of program eligibles reporting incomes on or 
just under the free meal eligibility cutoff point. In school year 1980-81, when free 
meal benefit guidelines were distributed to parents, 2.2 percent of free meal benefit 
eligibles reported annual incomes on or less than $200 under the free meal eligibility 
cutoff point. For school year 1981-82, when the free mekl guidelines were removed 
from the application materials, the figure remained constant at 2.2 percent. 

The new application form was shown to have a variety of effects on applicant 
income reporting, program eligibility and participation, and program costs to the 
Federal government. Eligibles using the new application form reported an average 
increase in income of more than $500 greater 'than program applicants using the 
prior-year application form. This finding strongly suggests that the new application 
form has been at least partially successful in preventing under-reporting of income 
on meal benefit applications. 

■ '■ : •• : .'.'•...<:• 28 ; ^ 



The propoptoivof ohU*eh PflMlvtog free lunches relative to school enrollment 
was approximately IQ percent lower In schools using the new application form than 
In Bohoote using the prior-year form, Similarly, the proportion of reduaed-prioe 
lunch beneficiaries was approximately 15 percent lower In new application schools 
than In prior-year application schools,. The reduction In program participation 
associated with the new appUoation form translates Into lower Federal subsidies, In 
schools using the.new appUoation form, Federal lunch program subsidies averaged 
52.5 oents per student enrolled per day. In contrast, the average dally Federal lunch 
subsidy per student was 56.8 cents for schools, using the prior-year appUoation form. . 

The hardship income deductions were, found to affect the eligibility status of 
less than 2 percent of program eligibles in school year 1980-81 in the Phase I SFAs. 
Associated with elimination of the hardship deductions was a slight decrease in 
overall program participation and a slight increase in the percentage of eligibles 
reporting Incomes on or near the reduced-price eligibility guideline, 

The two experimental quality assurance procedures had no additional impact on 
application reporting behavior, program eligibility or participation, or program costs 
in, the Phase I SFAs where they were used in conjunction with the new application 
form. However, in San Diego, when the quality assurance procedures were used in 
conjunction with the prior-year form, the procedures were associated with lower 
rates of program eligibility and lower Federal program costs. One hypothesis that 
may explain this finding is that the new appUoation form is a stronger treatment 
than the quaUty"assurance notices and therefore the notices are effective when used 
with the prior-year appUcation form' but have no additive effect when used with the 
new form. 





Phase 1 analysis has been dlreot^d toward analyzing the usefulness of the new 
Congresstonally-mandated application form, warning notices, and Income guidelines 
as quality assurance procedures, The analysis to date has produced useful 
Information. Many questions, however, rejoin. , 

• The new application form appears 'to have been at least partially 
successful in preventing income under-reporting and preventing ineligible 
individuals from obtaining program benefits. This success is evidenced by 
an increase in income^rjep'orted b'y applicants and a lower program 
eligibility and participation rate associated with the new application 

* form; Available data is insufficient for determining how ..much of the 
reduction in program eligibility and* participation resulted from the new 
form deterring ineligible individuals from applying or misreporting 
information on the. application and how much of the, reduction was the' 
result of the new form acting as a, barrier to eligible Individuals. Until i 
this issue is effective^ addressed,, the central question of Whether there 
is still a significant problem of applicant misreporting will be unanswered. 

• Audit warning notices ; when used with the new application form and 
without supporting fierce ment action, appear ineffective. 

• Ihcd me gu ideline : notices accompany ing the new application form (used in 

■ Phase I) appear inigilective. However, the possibility .exists that an 
effective guideEne notice could be developed based on an empirical study ; 

■ of reasons for applicant, ; misreporting. The in-home audits to be ; 
conducted later ; in Phase I will provide necessary background data on 
reasons for applicant nriisreporting., 




; APPENDIX A 
STATISTICAL PROCEDURES 

This appendix presents a summary of the statistical procedures used to estimate 
treatment group effeots, Treatment group effects were modeled through a system 
of/oontrast eodes defining, effeots assoolated with the three a priori hypotheses 
specified in section 3,2, The General Linear Model (QLM) procedure of the 
Statistical Analysis System (SAS) computer program was employed to estimate the 
normal equations. 

Models of treatment group effeots were estimated through a six-step process. 
First,. In the non-computerized SFAs, values of variables to be Included In the 
models were computed and their distributions examined to Identify outliers and 
distributional irregularities that Could violate underlying assumptions of the models. 
All outliers were examined for potential coding errors, and in several cases school 
district authorities were contacted to confirm reported values. Second, 1980/81 
data for dependent variables to be estimated for 1981/82 were examined 1>y 
treatment group to identify potential sampling anomalies that could bias treatment 
group effect estimates. No anomalies were found. Distributions of prior year 
program participation and eligibility variables'! are compared across treatment 
groups to detect any anomalies created by assignment of school%.to treatment 
groups. On the basis of simple chi-square and f-ratio tests, all prior year variables 
available to the study were unrelated to treatment group. Table A.l shows the 
results of these* tests. As can be seen, there were no pre-existing statistically 
significant differences between experimental groups on key variables. Third, 
ordinary least squares (OLS) models were estimated and the residuals examined to 
detect technical pathologies and outliers. When significant heterogeneity: of 
variance was found in the residuals of the program participation variables, an 
angular distribution transform was performed. Fourth, the models were then 
reestimated, removing icovariates and interaction terms that made no discernible 
■ ' > A.l >. - • * 
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hmlym ©f Vapinnee of Prior Year 
(NovtmbiP 1980) Variable Amm 
Treat mentQroupa 

Variable M p SignKibaiwe 



' Operating days 


3/88 


,053 


M 


Full prl««s breakfasts 


3/88 


,042 


M 




3/88 


XMi 

il l W M W 


.37 


Free breakfasts 


3/88 


,515 


,8V 


ruu price lunciies 




/ ,655 


,58 


' Reduced price lunches 


3/68 


V .384 


.77 


Free lunofrcjar 


3/68 . 


.130 


.04 


Average dally attendance 


3/68 


.485 


.71 


Total certlf led for free 
meals 


3/08 


.052 

' ' 'i 


.08 


Total certified for reduced 
price meals 


3/68 


.009 


.99 

* - 
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contribution to the * model and were not of substantive interest. Residuals were 
reexamined and at this point no technical pathologies were found. (If pathologies 
had persisted, an additional step would have been added to the analysis using 
jack-knife estimation procedures to minimize bias and increase estimate stability.) 
Fifth, models were independently estimated using weighted-least-squares (WLS) 
estimators derived under a finite population sampling model. (The OLS estimators 
0ere derived under classical, asymptotic sampling models.) Results *of the WLS and 
OLS estimates were compared and no statistically or substantively significant 
differences were discovered. Sixth, steps one through five were replicated for the 
computerized SFAs. 

A.j. The General Model 

The four treatment groups in Phase I SFAs were compared through a system of 
orthogonal linear contrasts as defined by t£e equation 

4 

C = Sa.y. ' (1) 

subject to the condition 

Z8j= 0 ; ' .* (2) 

Where: 4 

, a j is a value code for experimental group j; 

7j is the mean value of dependent variable y for experimental 
group j; , 

The magnitude and direction of C's 1 deviation from 0 is a measure of ,the 
hypothesized effect. For a ndndirectional hypothesis, statistical significance is a 
function of P(C f 0). For a directional hypothesis, statistical significance is , a 
function of either P(C<0) or P(C>0), depending on the expected direction of^he 
effect. Three linear contrasts were conducted corresponding to the three 
hypotheses specified in Section 3.2. Table A.2 defines the coding structure used. As 
can bejseen; thejsum of all the columns in Table A.2 is zero and the covariance of 
the columns is zero. .; ^ 

"... Contrast coding has several important advantages over classical ANO V A 
procedures in the present case. .Contrast coding handles problems of unbalanced 
designs (as occurs in estimates 'involving breakfast program * participation) more 
easily than does ANOV A. Contrast coding produces greater estimate precision in 
disaggregating effects of the new application form from the quality assurance 
procedure ^effects. Finally, contrast coding allows use of a priori hypotheses and 
directional significance tests that are often difficult with ANOV A. * . 



TABLE A.2: CONTRAST CODES, 



Hypothesis Contrast 



Treatment Group 


Hi 


H2 


. H3 


1 1981/82 Application r 


1/3 


-1 


0 ' 


2 1981/82 Application, with Warning 


1/3 


1/2 


-1 


3 1981/82 Application with Guideline 


1/3 


1/2 


1 , \v 


4 1980/81 Application 


-1 


0 


0 



Effect parameters associated with the hypothesized contrasts were estimated 
by the normal least squares equation: ; 



(3) 



where 



£is a vector of coefficients associated with an intercept term and a set of 
independent variables, including the three contrasts codes; 

X is a data matrix having all l's as its first column and individual values 
of the independent variables as its remaining columns; and 

Y is £ data vector of values of yj. ' 
The normal equations were estimated using the GLM procedure of SAS. 

As an exploratory measure, an alternative finite population sampling model was 
estimated. Under a finite population model, variables describing the population 
under study (in this tfaSe program applicants in Phase 1 schools) are seen as random 
realizations of a process operating in a larger super population. Therefore even if a 
complete census of program applications were studied in all participating schools, 
the total school value for any given variable is regarded as a random rather than a 
fixed value. As a random" variable, the sum of a given variable, 'Y\ in a school is 
subject to an error variance inversely proportional to the size of the finite 
population from which it Was drawn. For some of the variables estimated, the 
population base was total school enrollment; ■ for others, total applicants. To 
account for this error variance and thereby increase parameter estimate precision, a. 
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weighted least squares approach was used in which £w (y - y)2 is minimized and w 
is propprtiohal to the population base. The resulting normal equation is: 

(X t WX)" 1 X f WY ' (4) 
where W is a diagonal matrix of population base values. Comparisons of parameter 
estimates for weighted and unweighted models showed only marginal variations and 
no clear superiority of either model. For simplicity/ the results of the unweighted 
models are presented. ' 

A.2 Program Participation 

Rates of program participation for an individual school 'i f and program category 
7 were defined by the equation 

P ij = (Kij)/(£iXDi) (5) 

where jg the num ber of meals served in program category 'j 1 in school ! i f in 
November 1981. / 

E i is enrollment in school i in November 1981; and 

D i is the number of days meals were served in school i in November 1981 r 

Program participation was defined in this way to standardize for variations in 
school size and number of operating days. Problems of bias commonly associated 
with sample-based ratio estimators do not occur in this case because by definition 
the expected value of the number of meals served is zero if either enrollment or 
operating days is zero and the ratios are calculated on the basis of a census of the 
total relevant population and not a; sample. 

Because initial analysis revealed heterogeneity in model residuals, an angular 
(arc sin) transform was performed on participation rate variables. That is, in the 
analysis, rather than using 'P...as defined above, the following variant was used: 

Pij = arc sin ( v^ij) (6) 

• ftj has the advantages of an asymptotic normal distribution, greater stability than 
p ij and a (variance that is independent of Py.2/ Subsequent residual analysis 
revealed no technical pathologies. 



A.3 Variable Definitions 

Table A.3 presents the variable mnemonics and titles jlsed in the preliminary 

.-v . ' • . 

analysis. . 

Final Models V 

— ~ ' . ■ ., * V- 

Table A.4 presents results of the final models in tecms N of the statistical 

significance of, parameter estimates. Without elaborate manipulations, the raw 
parameter estimates and decomposed sums of square are unint;erpretable, therefore 
they have been excluded from the table. Contrast 1 and Contrast 2 correspond to 
hypotheses that predict direction treatment effects, thus except where indicate'd, 
one-tailed^t-tests were used to determine statistical significance. T Contrast 3 
corresponds to a non-directional hypothesis and as a result two-tailed t-tests were 
used. 



' NOTES ; " 

17 S Cohen ™rt P.mhgn. Ap plied Multiple Regression ^ Correlation s A n alysis for 
" the Beh avioral Sciences. Wiley & Sons/ New York, 19'/ b, pages lyo-^present 
the basic rationale and derivations for contrast coding of experimental data. 

2/ For an introductory discussion of angular transforms, see Y.M. Bishop, S.E. 
" Fienberg, and P.W. Holland. Discrete Multivariate Analysis, MIT Press, 
Cambridge, 1975, pages 366-368. 
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TABLE A.3: VARIABLE MNEMONICS 



Mnemonic 



Variable Title 



11/1980 
11/1981 



the lunch program 



FL1180 Rate of free lunch participation 11/1980 
FL1181 Rate of free lunch participation 11/1981 
RL1180 Rate of reduced-price lunch participation 11/1980 
RL1181 Rate of reduced-price lunch participation 11/1981 
PL1180 Rate of full price lunch participation 11/1980 
PL1181 Rate of full price lunch participation 11/1981 
FB1180 Rate of free breakfast participation 11/1980 
FB1181 Rate of free breakfast participation 11/1981 
RBI 180 Rate of reduced-prkie breakfast participation 
RB1181 Rate of reduced-price breakfast participation 
PB1180 . Rate of full price breakfast participation 11/1980 
PB1181 , Rate of full price breakfast participation 11/1981 
SFA School Food Authority / *">^ ' , 

CSTL80 Average daily per student Federal subsidy of the lunch pfogram 
11/1980 

CSTL81 Average daily per student Federal §ub^dy of 

n/1981 ^, ; '?;^f:; \ 

CSTB80 Average daily per student Federal subsidy of the breakfast program 

11/1980 • J * % 

CSTB81 " Average daily per student Federal subsidy of the breakfast program 

11/1981 / ' 4 

INC80 Average annual reported income for 1980 applicants 

INC81 Average anftual reported income for 1981 applicants 

INCHG Average annual change in reported income for 1980 applicants who 

reapplied in 1981 

SOURCE Number of income sources reported on 1981 application 
REAPPLY Proportion of 1980 program applicants who are also 1981 applicants in 
the same school 

THR80F Proportion of ^980,free meal recipients reporting an annual income on 

or less than $20&° under the free meal eligibility cutoff point 
THR81F Proportion of 1981 free meal recipients reporting an annual income on 

or less than $200 under the free meal eligibility cutoff point 
THR80R Proportion of 1980 reduced-p/ice meal recipients reporting an annual 

income on ot less than $200 under the free meal eligibility cutoff point 
THR8TR Proportion of 1981 reduced-price meal recipients reporting an annual 

income on or less than $200 under the free meal eligibility cutoff point 
APR80 Proportion of students enrolled approved for meal benefits in 1980 
APR81 ' Proportion of students enrolled approved for meal benefits in 1981 
CONTRAST! Treatment Group Coding corresponding to Hypothesis 1 in Section 3.2 
CONTRAST2 Treatment Group Coding corresponding to Hypothesis 2 in Section 3.2 
CONTRAST3 Treatment Group Coding corresponding to Hypothesis 3 in Section 3.2 



TABLE A.3: • STATISTICAL SIGNIFICANCE OF ESTIMATED EFFECT PARAMETERS FOR FINAL 
. MODELS (NONCOMPUTERIZED SF As) 
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DEPENDENT 
VARIABLE 






MAIN trrtLlo 








MODEL 
NUMBER 


'SF»<» 


Contrast 


Contrast 2*^ 


Contrast 3 


(3) C0VARIATES (3) 


NUMBER OF 
CASES 


1. 


FL1181 


.2962 


.0010 " ,f 


.9914(4) 


.6652 


FLU80 . 0001 
PL1180 .1846 






RL1181 


.22 96 


.0459 


.0690 


.7900 


RLllBO .0001 

FLUBO .3082 
Dillon ccdd 


72 


3. 


ni 1 1 01 

PL1181 

■ * 


.0001 




. JJ/O 


Q7QA 


PI 1180 .0001 
FL1180 .3483 


72 


4. 


FB1.181 


.6457 


.1434 (4) 


.1224 


.MJ/ 


v call on OAfil 
y rpiiou iUuui 




5. 


' RB1181 


' .0064 


.7529 (4) 


.0434 (4) 


.9241 


RBI 180 .0001 
coi Ion 

■ 


35(6) 


6. 


PB1181 


.2114 


AIIO (fit 

.4410 (5) 






pan an 0005 
FBU80 , .5590 


33 (61 




CSTL81 


.0001 


.0004 


.1749 


.6791 

* 


CSTL80 .0001 


72 


8. 


CSTB81 ; 


.7084 


.3885 (4) 


.0737 


.4064 


CSTBB0 .0001 


35 (6) 


9. 


SOURCE 


.0001 


(7) 


.4506 


.6451 


INC80 (9) .3733 


54(7) 


10. 


» INQIG(9) 


,2800 


.1629 


.9815 (4) 


.8895 


INC80 ( 9) .8333 


69(8) 




REAPPLY 


.0506 


.2026 


.4703 (4) 


Me A 

• 9264 


APRoU JJoO 




12. s 


THR81FI9) 

i 


.2017 


.3989 (4) 


.1843 


, .3686 


THR80F (9) .1232 
THR80R (9) .1739 


72 


• '"''?' - ' ■ 
13. ^ 


THR8lR(9) 


.0080 


.7503 (4). 


.0625 


.5071 


THRB0R (9) '.0598 
THR80F (9) .3060 


72 



MULTIPLE 
R SQUARE 
OF MODEL 



.9866 
.7319 

.9699 

.9593 
.9741 

.7362 

.9811 
.9550 
.5997 
j 1738 
.3.127 
.2615 

.3538 ; 



NOTES 

(1) Test based on F Ratio for type IV suns -of -squares breakdown. 

(2) unless otherwise indicated, probability based on one-tailed t-test. 

(3) Probability based on two-tailed t-test. 

(4) Estimated effect In opposite direction predicted by hypothesis, two-tailed t-test used. 

,(5) Hypothesis does not predict directional effect, two-tailed t-test ' 

(6) Three Phase I SFAs do not have breakfast programs and 13 schools in the remaining 6 SFAs.do not have breakfast 
programs. 

(7) Treatment Group 1 not Included In the model RPOTPflllV fMiU fimp 
(Irni^ and 1981 applications could not be matched In three schools OtOLp | AVHlLAsLc 
(caj&aij.er child applicants' likome not used in making estimates. ' 46 



7 

dependent 
variable: 



mm crrcuo 



.(2) 



Contrast V ' Contrast 2 1 *' Contrast 3 



(2) 



,(3)- 



COVARIATES 



(3) 



NUMBER Of 
CASES 



MULTIPLE 
R SQUARE 
OF MODEL 



FL1181 



.2962 . 0010 ,f .9914(4) 



.6652 



RLH81 


.2296 


.0459 


.0690 


.7900 


PLH81 


.0001 


.7187 (5) 


.3376 (5) 


,9298 


Fftllfll 
rOilOi 




1414 (4) 


1224 


* 

.5537 






7529 (41 


0434 U\ 


9241 


PB1181 


.2114 


.4318 (5) 


.3231 (5) 


.4695 






0004 


.1749 


.6791 


CSTBBl, 


.7084 


.3885 (4) 


.0737 


.4064 


SOURCE 


.0001 




.4506 


.6451. 


» INQIG(9) 


,2800 


.1629 


.9815 (4) 


.8895 


REAPPLY 


.0506 


.2026 


.4703 (4) 


.9264 


THR8lF(9) 


.2017 


.3989 (4) 


.1843 


i 

, .3686 


THR8lR(9) 


.0080 


.7503 (4). 


.0625 


.5071 



FU180 
PL1180 

RL1180 
FL1180 
PLH80 

PL1180 
FLU80 



.0001 
.1846 

.0001 
.3082 
.5544 

.0001 
.3483 



\ FB1180 .0001 



RB1180 
FB1180 

PB1180 
FBU80 

CSTL80 

CSTBB0 



.0001 
.3826 

.0005 
.5590 

.0001 
.0001 



INC80 (9) .3733 
INC80 (9) .8333 
.3386 



THR80F (9) .1232 
THR80R (9) .1739 

THR80R (9) ',0598 
THR80F (9) .3060 

I 
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72 
72 

35 (6) 
35 (6) 

35 (6) 

72 

35 (6) 
54(7) 

69 (8) 

69 (8) 
72 

72 



.9866 
.7319 

.9699 

.9593 
.9741 

.7362 

.9811 
.9550 
.5997 
;1738 

■3J27 
.2615 

.3538 



:st based on F Ratio for type IV suns -of -squares breakdown. 
Hess otherwise indicated, probability based on one-tailed t-test. 

lability based on two-tailed t-test. 

stiinated effect in opposite direction predicted by hypothesis, two-tailed t-test used. 

rpothesis does not predict directional effect, two-tailed t-test used. ' 

iree Phase I SFAs do not have breakfast programs and 13 schools In the remaining 6 SFAs do not have breakfast 

rograms • 

reatment Group 1 not Included In the model RFQT PflDV IMWm* 

980 and 1981 applications could not be matched In three schools WMp I A/ftfjJljrtJ.f 
oster child applicants' Income not used In making estimates. 
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" ■ ' . v' ■ ••' :'' ■ . ;.. : , APPLICATION 

*'■ ■■ "'■ ■ - .■ ' ■ ■■,■$■.:>* » . . 

'PARENTS: To ipplyifor ffee'and reduced pclice n^le for your children, fill out this form and return it to 
the school office.. . ^ ■ 

CHILDREN: Names and Grades bf Children for Whom Application is Made 

NAME 1 ' ' -■' — SCflOOt ~\ A .GUM 



ALL OTHER HOUSEHOLD MEMBERS : ■' .' ■ ^ ^ ^ ^ ^ ^ ^ 

HOUSEHOLD MEMBERS: ' List; all related or unrelated persons who live in your household and share living expenses j * 
or meals. ( Don't include children listed above) 

^SOCIAL SECURITY NUMBER: List the Social Security number of all adults. Adults are family members 18 years or 
older. 41 . , » . J ( . 

, ■ ; V koUSEHOLD MEMBERS / SOCIAL SECURITY NUMBER * 

* " ■ ■ * * * 
. ' r : . '" ; " ' : *=*5 



TOTAL: Total number of Household Members : 



HOUSEHOLD INCOME: Enter amount of current income before deductions, such as taxes and Social Security, in 
appropriate category. * If, you receive more than one check from any one of these sources, please indicate the 



total monthly amount received. 



Wages, Salary 



Social Security 



Public Assistance (Welfare) 
Ijnemp'loyment |__J_ 



TOTAL PER MONTH 

Child Support (Alimony) 
Pension or Retirement 
Other ' , 



TOTAL MONTHLY INCOME 



FOSTER CHILDREN:. In certain cases foster children are eligible for free or reduced price meals regardless of < 
their family's income. If you have foster children living :wUh you an* wish to apply for such meals for them,< 
please check here. I I ' , ' \ 

SIGNATURE /ADDRESS : l' hereby certify that all the above information is true and correct. I understand that 
this information is being given in connection with the receipt of Federal funds; that school officials may , 
verify information; anji that deliberate misrepresentation may be subject to prosecution under applicable State/ 
and Federal criminal statutes. ?■ 





... SIGNATURE OF ADULT FAMILY MEMBER 


DATE 




TELEPHONE 




t 


; PRINT NAME . 




ADDRESS 7 







0 

ERIC 



FOR SCHOOL USE ONLY 
/ V 7 Approved Free 

r 



Approved Reduced 
I I Denied Reason for Denial 
I -y p arent:3 Notified 



Date 



Determining Official 



Signature 
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Date 



48 



IEST COPY WmL; 



4 } 



u 



Dear Parent: * . . r v • 

The school which your child attends participate* In the National School Lunch Program. All students are 
encouraged to enjoy thia nutritious lunch each school day . Students may' purchase lunch for: ■ ^■^v'^: 1 -. 

Elementary, 85c; Secondary $1.00 v.V ; '''ig-r 

l-5c ' 

Extra milk may be purchased for: - . . ' ^ 



If your school participates In the School Breakfast Program, students may purchase breakfast for: 



, : — ; ^ rg ; 

Students from families whose Income Is at or below the level shown on the income guidelines scale may be 
eligible for either free mesls or me sis at a reduced price of AO cents for lunch and 30 cents for breakfast. 

To apply for free or reduced priced meals for your children at any time during the year, complete the^tt ached, 
application and return It to the school. If /during the school year, there are changes in your family size or 
substantial changes in your income, please report these changes to the school's determining official in order 
for appropriate eligibility' adjustments to be made.. 

- '■■ ,■'>'. ■' . 

* Verification of Current Family Income - The application now requires the name and Social Security 
number of all adult family members and specific Income 'information . Adults without Social Security 
numbers must indicate that they do not have one. School officials may verify the source and amount of 
Income as well as household composition. 

* Foster Children - In certain cases foster children are slso eligible, for these benefits. If you have 
foster children living with you and. you wteh to apply for such 'meals for them, please notify us or. 
Indicate it on the application. 

* Nondiscrimination - All students are treated the same regardlesa of ability to pay. In the operation 
of child feeding programs, no child will be discriminated sgsinst becauae of Srace, sex, creed, color 
or national origin. 

* Fair Hearing; * Within 10 days of receiving your application, the school will notify you whether or not 
your children are eligible. If you do not agree with the school's decision on your application, you 
may wish to discuss it with the school official. If you wish to review the decision further, you have 

'■) a right to a fair hearing. This can be done by calling or writing. 



(NAME OF HEARING OFFICIAL) (ADDRESS) q C" (PHONE NO ; ) 

INCOME ELIGIBILITY GUIDELINES, School Year 1981-82 ■" * 

FAMILY SI22 ■ _ 

1 — ' : " . ' > $ 7,970 

2 . ;~. • 10,530 . 

3 — ~ . 13.080 

4' 15,630 

5 „ ? . ; : . 18,190 

6 — : 20,740 

7 £ — • : 23.290 

3—- ----- : : /;-;'• 25,840 

Each additional family member " m ^2,550 ^ 

If we can be of further assistance, please do not hesitate to contact us. ' 

Sincerely, 



NAME 

Food Service Director 



TITLE 



; (Rev. 3-81) 
SFL/16 
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